(19) 



J 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 




(11) 



EP 1 255 404 A1 



(12) 



EUROPEAN PATENT APPLICATION 

published in accordance with Art. 158(3) EPC 



(43) Date of publication: 

06.11.2002 Bulletin 2002/45 

(21) Application number: 01904374.4 

(22) Date of filing: 09.02.2001 



(51) int CI7: H04N 5/44, H04N 5/76, 
H04N5/91, H04B1/16, 
G11B 20/10 

(86) International application number: 
PCT/JP01/00949 

(87) International publication number: 

WO 01/060057 (16.08.2001 Gazette 2001/33) 



(84) Designated Contracting States: 

AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 
MC NL PTSE TR 

Designated Extension States: 
AL LT LV MK RO SI 

(30) Priority: 10.02.2000 JP 2000034207 

(71) Applicant: TOKYO BROADCASTING SYSTEM 
INC. 

Tokyol 07-8006 (JP) 

(72) Inventors: 

• TAKAHASH), Toshiaki, 
TOKYO BROADCASTING SYST. INC. 
Minato-ku, Tokyo 107-8006 (JP) 



• SANO, Takeshi, 

TOKYO BROADCASTING SYST. INC. 
Minato-ku, Tokyo 107-8006 (JP) 

• SUGIURA, Fuyuhiko, 

TOKYO BROADCASTING SYST INC. 
Minato-ku, Tokyo 107-8006 (JP) 

• SASADA, Masaako, 

TOKYO BROADCASTING SYST. INC. 
Minato-ku, Tokyo 107-8006 (JP) 

(74) Representative: Grunecker, Kinkeldey, 

Stockmair & Schwanhausser Anwaltssozietat 
Maximilianstrasse 58 
80538 Munchen (DE) 



(54) RECEIVING TERMINAL DEVICE AND CONTROL METHOD THEREFOR 



(57) A receiving terminal device which provides 
viewers with broadcasting radio waves from broadcast- 
ing stations as programs and in which, when a program 
switching is executed while a first program is being pro- 
vided and a program is switched to a second program, 
the second program is provided and video recording of 
the first program is started, and when the program is 
switched back to the first program after the program is 
switched to the second program, the recorded first pro- 



gram is provided. Then, the receiving terminal device, 
when the first program is selected again, executes 
catch-up reproduction of the recorded first program in 
the manner that the position of recording and reproduc- 
ing the first program catch up with a position in which 
the first program is being broadcast by the broadcasting 
station. Thus, even in case that a switching of a program 
selection is executed, a broadcasting service without 
failure of program contents viewing can be realized. 
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Description 

Technical Field 

[0001] This invention relates to output devices and re- 
ceiving terminal devices for use in digital broadcasts. 

Background Art 

[0002] In general television broadcasting services, 
viewers receive broadcasting waves sent from broad- 
casting stations by receiving terminal devices and view 
programs which are expressed as audio/video informa- 
tion. 

[0003] That is, when a program is a real time type pro- 
gram, a receiving terminal device decodes and repro- 
duces in real time a selected stream from those trans- 
mitted from a broadcasting station thereby providing vid- 
eo/audio information to a viewer. When a viewer switch- 
es the channel, the receiving terminal device decodes 
and reproduce a newly selected stream instead of the 
stream being decoded and reproduced previously. 
[0004] In the case where the real time type program 
is viewed in the manner as described above, a viewer 
cannot view a part of a program content during the pro- 
gram being viewed is interrupted and switched to anoth- 
er program to view another program even when the 
channel is switched back to view again the program orig- 
inally viewed. Therefore, when the viewer wishes to 
view a certain program consecutively, the viewer deals 
with this by video recording the program and viewing it 
afterwards. 

[0005] However, it is very troublesome to operate for 
video recording every time of switching the channel and 
after all; the viewer has no other way but records the 
program from the beginning. Further, in case that the 
program was recorded from the beginning, it is difficult 
forthe viewerto determine the time point to play, so that 
the viewer has to search the point to resume by con- 
ducting high speed play reproduction. 
[0006] Further, implementation of digital broadcasting 
services multichannel broadcast and various services 
will be provided. For example, a service in which a 
broadcasting station controls receiving terminal devices 
of the viewers to switch a program by using a transmis- 
sion stream becomes possible. In this case, when it is 
controlled to switch the real time program to another pro- 
gram and then to switch back to the former program, 
viewing of the program is obstructed if a failure of the 
program content to view is cased, thus it is not satisfac- 
tory as a broadcasting service. 
[0007] Consequently, it is an object of the present in- 
vention to provide a broadcasting service which does 
not cause failure of viewing of program contents even if 
the program selection is switched. 
[0008] Particularly it is an object of the present inven- 
tion to provide a broadcasting service in which, when a 
real time program which is being viewed by a viewer is 



interrupted to switch to another program and then 
switched back to the former program, the program is re- 
sumed from the program content at which the program 
is interrupted. 

5 [0009] Further, it is an object of the present invention 
to provide a broadcasting service in which, when a real 
time program which is being viewed by a viewer is in- 
terrupted to switch to another program and then 
switched back to the former program, viewing is re- 

10 sumed from the program content at which the program 
is interrupted and the time difference caused therefrom 
is solved gradually, so that the viewer does not feel un- 
comfortable. 

15 Disclosure of the Invention 

[001 0] An aspect of the invention is a receiving termi- 
nal device which provides broadcasting waves from 
broadcasting stations to viewers as programs, in which, 
when program switching is executed while a first pro- 
gram is being provided to switch to a second program, 
the second program is provided and a video recording 
of the first program is started, and when the switched 
second program is switch back to the first program, the 
recorded first program is provided. Switching between 
programs may be made by control information which is 
transmitted from a broadcasting station or by voluntary 
switching (selection of stations) by a viewer. 
[0011] Then, when thefirst program is selected again, 
the invention executes catch-up reproduction in the 
manner such that the position of reproducing the first 
program catches up with the position in which the first 
program is being broadcasted by a broadcasting station. 
[0012] Thus, it becomes possible that the viewer, 
even while a program is being switched, views afterward 
contents of the program which is being broadcasted by 
the broadcasting station. Further, it becomes possible 
that the viewer views contents of the program with a 
smallertimethan the time during which another program 
was being switched. 

[0013] Specifically, the invention is a receiving termi- 
nal device which selects a particular stream from the re- 
ceived streams and reproduces the selected stream as 
video/audio information, characterized in that, when a 
second stream is selected while a first stream is being 
filtered through, the device stores a stream in the first 
stream following the time point at which the second 
stream is selected and reproduces the selected second 
stream. 

[0014] Here, the receiving terminal device is charac- 
terized in that, when the second stream is selected and 
thereafter the first stream is selected again, the stored 
stream is reproduced. 

[0015] Further, the receiving terminal device is char- 
acterized in that it obtains reproduction start position in- 
formation from the received stream and reproduces the 
stored steam from the reproduction start position. Thus, 
the receiving terminal device becomes possible to re- 
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produce the stored stream from an arbitrary position. 
[0016] Furthermore, the receiving terminal device is 
characterized in that it obtains time management infor- 
mation at the selected time point and reproduces the 
stored stream from the position specified by the ob- 
tained time management information. 
[0017] On reproduction of the stored stream, the re- 
ceiving terminal device is characterized in that the 
stored stream is reproduced in the manner such that the 
reproduction position of the stored steam in a reproduc- 
tion time axis catches up with the reproduction position 
of the first stream in a real time axis. Further, it is pref- 
erable that the receiving terminal device displays on a 
screen a time difference information between the repro- 
duction position of the stored steam in the reproduction 
time axis and the reproduction position of the first stream 
in the real time axis. 

[0018] In addition, in order for the receiving terminal 
device to reproduce the stored stream in the manner 
such that the reproduction position of the stored stream 
in the reproduction time axis catches up with the repro- 
duction position of the first stream in the real time axis, 
the stored stream may be reproduced in fast play repro- 
duction. Further, the receiving terminal device may ex- 
ecute thin-out processing on the stored stream. 
[0019] As for this thin-out processing, there is a meth- 
od in which the thin-out processing is applied to a par- 
ticular frame data to be generated based upon the 
stored stream. Further, there is a method in which a par- 
tial stream corresponding to a predetermined program 
content in the stored stream is processed to be thinned 
out. Furthermore, a partial stream in the stored streams 
corresponding to a program content other than a pro- 
gram content which is selected in accordance with user 
information inputted in advance may be processed to 
be thinned out. Moreover, when the stored stream con- 
tains n partial streams corresponding to the predeter- 
mined program content, n-1 partial streams may be 
processed to be thinned out. 
[0020] In addition, the partial stream to which the thin- 
out processing was applied may be stored and read out 
according to need to be reproduced. 
[0021] Further, the receiving terminal device is char- 
acterized by storing the first stream until the reproduc- 
tion position of the stored stream in the reproduction 
time axis catches up with the reproduction position of 
the first stream in the real time axis. 
[0022] Furthermore, the receiving terminal device is 
characterized in that, when the first stream is selected 
again after the second stream is selected, the selected 
first stream is reproduced on a first window and the 
stored steam is reproduced on a second window. 
[0023] Here, the receiving terminal device is charac- 
terized in that a partial stream corresponding to the first 
program content in the stored stream is reproduced on 
the second window. Further, the receiving terminal de- 
vice is characterized in that a partial stream correspond- 
ing to the second program content in the stored stream 



is reproduced in fast play reproduction on the first win- 
dow. 

[0024] The program switching is executed by control- 
ling the receiving terminal device by a control signal 
5 which is sentfrom a broadcasting station. Further, it may 
be executed by operation of the viewer which is given 
through an input unit. Furthermore, it may be executed 
in accordance with schedule information which is set 
based upon program guide information (EPG). 
w [0025] Moreover, the invention is a receiving terminal 
device characterized by comprising a receiving unit for 
receiving a transferred stream, a memory unit for storing 
the received stream, an accepting unit for receiving pro- 
gram selection information for selecting a program to be 
1* provided to a viewer, an selecting unit for selecting a 
particular stream in the received streams based upon 
the program selection information, an output unit for out- 
putting video/audio information based upon the stream 
to be transmitted, and a control unit for controlling in the 
manner such that the program selection information for 
selecting the second stream is received while the first 
stream is being filtered through, a stream in the first 
stream following the time point at which the second 
stream is selected is outputted to the memory unit. 
[0026] Further, the invention is a method for providing 
a program characterized by comprising a step for 
switching to a second program while a first program is 
being provided, a step for video recording of the first pro- 
gram while the switched second program is being pro- 
vided, a step for switching back to the first program while 
the switched second program is being provided, and a 
stage for proving the first program which is being record- 
ed. 

[0027] The method for providing a program is charac- 
terized in that, when the first program is switched back, 
catch-up reproduction of the recorded first program and 
is executed in the manner such that the position of re- 
cording and reproducing the first program catches up 
with the position in which the first program is being 
broadcasted by a broadcasting station. 
[0028] Specifically, the invention is a control method 
of a receiving terminal device which selects a particular 
stream in the received streams and reproduces the se- 
lected stream as video/audio information, characterized 
in that, when a second stream is selected while a first 
stream is being filtered through, a stream in the first 
stream following the time point at which the second 
stream is selected and the selected second stream is 
reproduced. Then, the control method of the receiving 
terminal device is characterized in that, when the first 
stream is selected again after the second stream is se- 
lected, the stored stream is reproduced. 
[0029] The invention can be practiced as a program 
which realizes a predetermined function in collaboration 
with a hardware such as a computer, etc., or a recording 
medium recording the program. 
[0030] In the present specification, means does not 
simply mean physical means but also includes a case 
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in which a function that the means has is realized by 
software. Further, a function that one means has may 
be realized by two or more physical means, and func- 
tions of two or more means may be realized by one 
physical means. 

Brief Description of the Drawings 

[0031] Fig. 1 is a block diagram showing a structure 
of a broadcasting station side transmission system ac- 
cording to a first embodiment. 
[0032] Fig. 2 is a block diagram showing a structure 
of a receiving terminal device according to the first em- 
bodiment. 

[0033] Fig. 3 is a block diagram showing a structure 
of a stream controlling unit according to the first embod- 
iment. 

[0034] Fig. 4 is a schematic diagram for explaining re- 
lation of a real time axis and a reproduction time axis. 
[0035] Fig. 5 is a flow chart for explaining an operation 
of the stream controlling unit according to the first em- 
bodiment. 

[0036] Fig. 6 is a flow chart for explaining an operation 
of the stream controlling unit according to the first em- 
bodiment. 

[0037] Fig. 7 is a view for explaining one example of 
program switching when a program is viewed as a 
stream. 

[0038] Fig. 8 is a view showing transition of a screen 
which is displayed on a display. 
[0039] Fig. 9 is a block diagram showing a structure 
of a stream controlling unit according to a second em- 
bodiment. 

[0040] Fig. 1 0 is a block diagram showing a structure 
of a stream controlling unit according to a third embod- 
iment. 

[0041] Fig. 1 1 is a flow chart for explaining an opera- 
tion of a catch-up reproduction processing unit accord- 
ing to the third embodiment. 

[0042] Fig. 1 2 is a view for explaining one example of 
program switching. 

[0043] Fig. 13 is a view showing an example of a 
structure of a screen which is displayed on a display. 
[0044] Fig. 14 is a block diagram showing a structure 
of a stream controlling unit according to a fourth embod- 
iment. 

[0045] Fig. 15 is a block diagram showing a structure 
of a stream controlling unit according to a fifth embodi- 
ment. 

[0046] Fig. 1 6 is a view for explaining one example of 
program switching. 

Best Mode for Carrying Out the Invention 

[0047] Then, embodiments of the present invention 
will be explained with reference to the drawings. 
[0048] In the invention, it is premised on mainly digital 
broadcasting services. In digital broadcasting services, 



video and/or audio information and data information can 
be multiplexed to be delivered to viewer. As a technol- 
ogy for realizing such digital broadcasting services, 
MPEG2 standard is known. In the following embodi- 
5 ments, it will be explained with MPEG2 standard as an 
example. 

[First Embodiment] 

10 [0049] The present embodiment is characterized in 
that, in case that a receiving terminal device is controlled 
in accordance with an instruction from a broadcasting 
station and after a real time program which is viewed by 
a viewer is interrupted once and switched to another 
15 program, it is returned to the former program, viewing is 
resumed from the program content at the time point of 
the interruption. 

[0050] In addition, in the following explanation, it is as- 
sumed that the real time program is a program relating 
to normal television broadcasting and a program to be 
changed is a program relating to data broadcasting in 
the same transport stream. Further, it is assumed that 
contents data of a program relating to data broadcasting 
is transferred by Carousel system. 
[0051] Fig. 1 is a block diagram showing a structure 
of a broadcasting station side transmission system ac- 
cording to the embodiment. A video/audio generating 
device 101 outputs video data (image data) and audio 
data (sound data) which are sent out from a video cam- 
era and a video server which are not shown in the figure 
to a video/audio data encoder (hereinafter, referred to 
as "video/audio data encoder") 1 03. In the specification, 
it is assumed that the video data and the audio data are 
handled as one data as long as it does not stick to it 
particularly, and this is called as video/audio data. A data 
broadcasting generating device 102 outputs contents 
data for use in data broadcasting (hereinafter referred 
to as "data broadcasting data") to a data broadcasting 
data encoder 1 04. The data broadcasting data compris- 
es text data, image data (still picture/motion picture da- 
ta), audio data, script (control program) and display ob- 
ject data and so on. In addition, the motion picture data 
has the same meaning as the video data but is used as 
such a meaning for distinguishing it from video data in 
television broadcasting. The data broadcasting data is 
produced by use of a not shown authoring terminal sys- 
tem based upon the video/audio data and other digital 
data and stored in a file server and so on. 
[0052] A broadcasting management device 1 03 gives 
an instruction for sending the video/audio data and the 
data broadcasting data to the video/audio generating 
device 1 01 and the data broadcasting generating device 
102, in accordance with information relating to program- 
ming. The broadcasting management device 103 also 
sends out various information necessary for an opera- 
tion for providing programs to a control information out- 
put device 107. 

[0053] The video/audio data encoder 1 04 compress- 
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es and encodes the video/audio data which is sent out 
from the video/audio generating device 101 by control 
of the broadcasting management device 1 04 in accord- 
ance with MPEG2 video and MPEG2 audio to generate 
a video/audio stream. The video/audio data encoder 
104 sends out the generated video/audio stream to a 
multiplexing device 1 08 in PES format. The data broad- 
casting data encoder 1 05 compresses and encodes the 
data broadcasting data which is sent out from the data 
broadcasting generating device 1 02 by the control of the 
broadcasting management device 1 03 to generate a da- 
ta broadcasting stream. The data broadcasting data en- 
coder 105 sends out the generated data broadcasting 
data to the multiplexing device 108 in a section format. 
[0054] An event message output unit 1 06 sends out 
an event message relating to the data broadcasting data 
which is sent out from the data broadcasting generating 
device 102 to a control information output device 17. 
The event message is, as described later, data which is 
interpreted by a receiving terminal device of a viewer, 
and the receiving terminal device executes processing 
according to this event message. That is, taking a look 
from a broadcasting station side, the broadcasting sta- 
tion can control the receiving terminal device of viewer 
by using the event message. The control information 
output device 107 generates various information which 
is sent out from the broadcasting management device 
103 and control information based upon the event mes- 
sage which is sent out from the event message output 
unit 106. The control information is defined as PSI/SI 
(Program Specific Information/Service Information). 
The control information output device 1 07 sends out the 
control information to a multiplexing device 108 in the 
section format. 

[0055] The multiplexing device 108 multiplexes the 
video/audio stream in the PES format and the data 
broadcasting stream and the control information in the 
section format to generate a MPEG2-transport stream 
(hereinafter referred to as "multiplexed stream"). The 
multiplexing device 108, on the occasion of generating 
the multiplexing stream, superposes furthermore time 
management information and so on according to need 
and executes a scramble concerning information to be 
kept as a secret. The multiplexing device 1 08 sends out 
the generated multiplexed stream to a digital modulating 
device 1 09. 

[0056] The digital modulating device 109 digitally 
modulates the multiplexed stream by a predetermined 
modulation system and sends out this to a transmitting 
device 110. As the predetermined modulation system, 
for example, 8PSK system, OFDM system and so on 
are exemplified. The transmitting device 110 transmits 
the digitally modulated stream as a broadcasting wave 
signal toward a relay station, for example, a broadcast- 
ing satellite. 

[0057] Fig. 2 is a block diagram showing a structure 
of the receiving terminal device according to the embod- 
iment. The receiving terminal device, typically, has a re- 



ceiving function, a storing function, a communicating 
function and a presenting function. In the following, it 
will be explained about the receiving function and the 
storing function and the presenting function closely re- 

5 lating to the embodiment, out of these functions. In ad- 
dition, such functions of the receiving terminal device 
can be realized by a hardware in mutual cooperation of 
a hardware and a software (program). 
[0058] Atuner201 selects a broadcasting wave signal 

10 corresponding to a particular transponder in the broad- 
casting wave signals which are supplied from a not- 
shown antenna and sends out it to a digital demodulat- 
ing circuit 202. 

[0059] The digital demodulating circuit 202 demodu- 

15 lates the broadcasting wave signal which was selected 
and sent out by the tuner 201 , and after a predetermined 
error correction processing is executed, sends out it to 
a demultiplexer 203. The signal which is sent out from 
this digital demodulating circuit 202 corresponds to the 

20 above described multiplexed stream. 

[0060] The demultiplexer 203 separates the multi- 
plexed stream to be sent out and selects a particular 
video/audio stream as the need arises and sends out it 
to a stream controlling unit 204. The demultiplexer 203, 

25 as to the separated data broadcasting stream , sends out 
it to a data broadcasting decoder 205b. The demultiplex- 
er 203 obtains PID (Packet Identifier) corresponding to 
a program to be selected and separates a stream in ac- 
cordance with this PID. The demultiplexer 203 selects 

30 a video/audio stream, based upon a program selection 
operation of a viewer and program switching control by 
the event message. The demultiplexer 203, when it ex- 
ecutes program switching processing, informs the 
stream controlling unit 206 of it. 

35 [0061] An input unit 204 is an input device which is 
operated by a viewer and typically, corresponds to a re- 
mote controller and an operation panel which is formed 
in a main program of the receiving terminal device. 
When a viewer wishes to select a desired program and 

*o to execute an interactive operation to the program view- 
ing the viewer, can give an instruction to the receiving 
terminal device through the input unit 204. The program 
selecting information which is supplied from the input 
unit 204 is sent out to a program selecting unit 205. 

45 [0062] The program selecting unit 205 receives the 
program selecting information which is sent out from the 
input unit 204 and sends out an instruction to the demul- 
tiplexer 203 so that a program is selected in accordance 
with the program selecting information. The demulti- 

so plexer 203, when it receives the instruction from the pro- 
gram selecting unit 205, selects a video/audio stream 
corresponding to a program to be selected. 
[0063] A stream controlling unit 206 controls an input/ 
output path of the video/audio stream to be sent out to 

55 the decoder side. That is, the stream controlling unit 206 
controls whether the video/audio stream which is sent 
out from the demultiplexer 203 is sent out to a decoder 
207 or to a storing device 208, and further, controls so 
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as to send out the video/audio stream which was stored 
in the storing device 208 to the decoder 207 instead of 
the video/audio stream which is sent out from the de- 
multiplexer 203. 

[0064] The decoder 207 has a video/audio data de- 
coder (hereinafter referred to as "VA decoder") 207a 
and a data broadcasting decoder 205b. The samefigure 
shows only one VA decoder 207a but a plurality of de- 
coders may be disposed. The VA decoder 207a de- 
codes the video/audio stream to be sent in accordance 
with MPEG2 video and MPEG2 audio. The VA decoder 
207a sends out the decoded video/audio stream to an 
output controlling unit 21 0. On the other hand, the data 
broadcasting decoder 207b decodes the data broad- 
casting stream and sends out contents data which is ob- 
tained by this decoding once to the storing device 208, 
and reads out the contents data from the storing device 
208 in accordance with a predetermined presenting tim- 
ing, and sends out it to a presentation engine 209. The 
data broadcasting decoder 207b also interprets an 
event message which is obtained by the decoding and 
executes processing based upon the event message. 
That is, the data broadcasting decoder 207b executes 
processing such as switching a program to be provided 
to a viewer, in accordance with the event message 
which is sent from a broadcasting station. 
[0065] The storing device 208 has a video/audio 
stream memory 208a for storing video/audio streams 
and a contents data memory 208b for storing contents 
data. The storing device 208, typically comprises a hard 
disk device, a RAM and so on, and it is preferable that 
it is of a built-in type one which is impossible to be re- 
moved by a user. In addition, in the same figure, one 
storing device 208 is configured by including the video/ 
audio stream memory 208a and the contents data stor- 
ing device 208b but may be configured by devices which 
are respectively independent physically. For example, 
the video/audio data storing device 208a may be con- 
figured by a hard disk device and the contents data stor- 
ing device 208b may be configured by a flash memory 
device. Further, a cache function may be provided as 
the need arises. 

[0066] A presentation engine 21 0 generates a screen 
data for use as a data broadcasting screen (hereinafter 
referred to as "data broadcasting screen data") accord- 
ing to a predetermined display format, based upon con- 
tents data which is sent out from the data broadcasting 
decoder 207b. A script which is contained in the con- 
tents data is also interpreted and executed by the pres- 
entation engine 209 to be formed as the data broadcast- 
ing screen data. The presentation engine 209 sends out 
the generated screen data to the output controlling unit 
210. 

[0067] The output controlling unit 21 0 synthesizes the 
video/audio data to be sent and the data broadcasting 
screen data, and converts it into an output signal (an 
image signal and a sound signal) which is suited to an 
output device and outputs it. In this figure, as the output 



device 211 , it is configured by a display (monitor) 211a 
and a speaker 21 1b. 

[0068] Fig. 3 is a block diagram showing a functional 
structure of the stream controlling unit 206 according to 

5 the embodiment. As described above, a basic function 
of the stream controlling unit 206 is to select a input 
source and a destination of the video/audio stream to 
be sent to the decoder side. In the embodiment, further- 
more, when reading out the stored video/audio stream 

10 and sending out it, the stream controlling unit 206 has 
another function to catch up concerning the video/audio 
stream and execute reproduction processing. 
[0069] A program switching monitoring unit 301 mon- 
itors whether a program which is provided to viewers is 

15 switched or not, and in case that it is judged that a pro- 
gram was switched, controls a selector 302 according 
to a state of program switching. The selector 302 selec- 
tively switches destinations, based upon control of the 
program switching monitoring unit 301 . The state of pro- 

20 gram switching is defined here as follows. 

(1 ) Initial State: a state after which a particular pro- 
gram A is selected but the program A was not se- 
lected in the past as after power was turned on. 

25 (2) First State: a state after which is switched to a 
program B in the initial state. 
(3) Second State: a state after which is switched 
again to the program A in the first state. That is, it 
is a state after which the program A is switched to 

30 the program B and then returned again to the pro- 
gram A. This may also include a case in which pro- 
gram B is switched first to a program C and then 
returned to the program A. 

35 [0070] The program switching monitoring unit 301 , in 
order to recognize which state is a state of program 
switching, when it receives a notice informing that a pro- 
gram was switched (selected), refers to an original pro- 
gram memory 303 which stored program number of the 

40 program which was being viewed until just before. That 
is, the program switching monitoring unit 301 , if a value 
which is stored in the original program memory 301 is 
null, determines that a state of program switching was 
transferred from the initial state to the first state, and 

*5 further, when program number of the program A is al- 
ready stored in the original program memory unit 301 
and it receives a notice informing that the program A 
was selected, determines that it was transferred from 
the first state to the second state. 

50 [0071] Thus, the program switching monitoring unit 
301 , in case of the initial stage, gives an instruction to 
send out the video/audio stream which is inputted from 
the demultiplexer 203 directly to the VA data decoder 
207, and in case of the first state, gives an instruction 

55 to send out the video/audio stream directly to the storing 
device 208, and in case of the second state, gives an 
instruction to send out the video/audio stream through 
a real time reproduction position obtaining unit 304 to 
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the storing device 208. 

[0072] The original program memory unit 303 stores 
program number of a program which was viewed until 
just before as stated above. Further, when the program 
A is switched to the program B, and then to the program 
C and thereafter returned to the program A, a program 
switching history may be stored. In this case, the pro- 
gram switching monitoring unit 301 may judge whether 
the program A exists in the history or not. 
[0073] The real time reproduction position obtaining 
unit 304 obtains a position to be received and repro- 
duced in a real time axis of the video/audio stream which 
is received in real time and sent out through the demul- 
tiplexer 203. Meanwhile, the read-out reproduction po- 
sition obtaining unit 305 obtains a position to be repro- 
duced in a reproduction time axis of the video/audio 
stream to be read out from the storing device 208. 
[0074] More specifically, the real time reproduction 
position obtaining unit 304 obtains sequence number of 
a packet which configures the video/audio stream which 
is being received, and regards it as a reproduction po- 
sition on a real time axis. Further, the read-out repro- 
duction position obtaining unit 305 obtains sequence 
number of a packet which configures the video/audio 
stream which is being read out from the storing device 
208, and regards it as a reproduction position on a re- 
production time axis. Here, in the context that receiving 
and reproducing are executed in synchronous with an 
actual transmission in a broadcasting station side, this 
is called as a reproduction in the real time axis, and in 
the context that reproducing is executed without syn- 
chronizing with the actual transmission of the broadcast- 
ing station side after the stream is one stored, this is 
called as reproduction in the reproduction time axis. 
[0075] Fig. 4 is a schematic diagram for explaining re- 
lation of the real time axis and the reproduction time ax- 
is. This same figure shows that in the real time axis, a 
stream of the program A and a stream of the program B 
are transferred from the broadcasting station side. Here, 
assuming that while the program A is being viewed, it 
was switched to the program B at a program switching 
time point E1 , and thereafter, at a program switching 
time point E2, the program B is switched again to the 
program A, the program content after the program 
switching time point E1 is made to be viewed. Now, sup- 
posing that the program switching time point E2 is P1 in 
the real time axis and time passes over from P1 to P2, 
the receiving terminal device is made to be reproducing 
a stream which is originally at a position of P2. However, 
a reproduction position of the stream which is viewed in 
fact is P2' in the reproduction time axis. Difference of P2 
in this real time axis and P2' in the reproduction time 
axis is called as a time difference. Further, viewing in a 
state that this time difference occurs is called as a time 
difference viewing. Then, as described later, the receiv- 
ing terminal device in the embodiment reproduces the 
stored stream so as to solve the time difference of this 
P2 and P2'. That is, time difference D right after being 



returned to the program A is: 

D = P2 - P2' > 0 

5 

but, by continuing the viewing of the program A, the 
stored stream is caught up and reproduced so that at a 
certain time point, it becomes: 

w 

D = P2 - P2' = 0. 

[0076] Returning to Fig. 3, a time difference detecting 
unit 306 obtains a time difference between a reproduc- 
es tion position of a video/audio stream in the real time axis 
and a reproduction position of a video/audio stream in 
the reproduction time axis, and informs a catch-up re- 
production processing unit 307 of the time difference. 
The time difference detecting unit 306, at a time point 

20 that it judged that the time difference was solved, con- 
trols the selector 302 so as to directly send out the video/ 
audio stream which is sent out from the demultiplexer 
203 to the VA data decoder 207a. 
[0077] The catch-up reproduction processing unit 307 

25 executes a catch-up reproduction processing to the vid- 
eo/audio stream to be read out, in such a manner that 
a reproduction position of the video/audio stream which 
is read out from the storing device 208 catches up with 
a position in case that the video/audio stream which is 

30 sent out from a broadcasting station in the real time axis 
is reproduced. As for the catch-up reproduction 
processing, for example, a thin-out processing concern- 
ing the video/audio stream is exemplified. Even in case 
that the thin-out processing is executed, it is preferable 

35 that the processing is made so as not to give uncomfort- 
able feeling to viewers. Therefore, in the catch-up repro- 
duction processing unit 307 in the embodiment, the 
catch-up reproduction is executed in the manner that 
only video/audio stream corresponding to a main pro- 

40 gram content in the video/audio streams passes through 
and a program content (local event) other than the main 
program content, for example, a partial stream corre- 
sponding to a commercial program is revoked. Determi- 
nation to a local event is executed with reference to a 

45 local event information table LIT (Local event Informa- 
tion Table) . Further, the catch-up reproduction process- 
ing unit 307 may detect commercial confirmation infor- 
mation and a predetermined description (descriptor) to 
revoke a partial stream (packets) corresponding to a 

so commercial program. 

[0078] As for the catch-up reproduction processing, 
in addition to processing for thinning out a particular pro- 
gram content itself, so-called trick plays including a fast 
play reproduction, a frame drop reproduction and so on 

55 are exemplified. These are realized by thinning out se- 
quentially I, P and B picture frames in the MPEG2 stand- 
ard from the B picture frame. 

[0079] It is preferable that such catch-up reproduction 
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processing is executed during decoding the packet by 
the VA data decoder 207a. In this case, the catch-up 
reproduction processing unit 307 is configured as a part 
of the VA data decoder 207a. If the VA data decoder 
207a executes such catch-up reproduction processing, 
it is configured so as to perform trick play control, while 
the VA data decoder 207a is receiving the time differ- 
ence information from the time difference detecting unit 
306. 

[0080] Further, as for the sound stream, it may be 
thinned out in a block unit, in order to synchronize video 
with audio. In this case, in the same way as normal re- 
production of MPEG-TS, PCR (Program Clock Refer- 
ence) may be used, in order to synchronize video with 
audio. 

[0081] Fig. 5 is a flow chart for explaining an operation 
of the stream controlling unit 206 according to the em- 
bodiment. In this figure, the program switching monitor- 
ing unit 301 monitors whether a program is switched or 
not, that is, whether there is a notice informing that a 
program was changed from the demultiplexer 203 or the 
output controlling unit 210 (STEP 501). The program 
switching monitoring unit 301 refers to the original pro- 
gram memory 303 and judges whether a value of the 
original program memory 303 is null or not (STEP 502). 
That is, the program switching monitoring unit 301 judg- 
es whether a program switching state is in the initial 
state or not. When the program switching monitoring 
unit 301 judges that the value is null, it controls the se- 
lector 302 to send out a video/audio stream which is di- 
rectly sent out from the demultiplexer 203 to the VA data 
decoder 207a(STEP 503). Thus, the selector 302 is 
made to select a terminal a in Fig. 3. The program 
switching monitoring unit 301 judges whether a program 
which was switched is a particular program or not (STEP 
504), and in case that it is judged to be the particular 
program, writes its program number in the original pro- 
gram memory 303 (STEP 505). The reason that whether 
it is the particular program or not is judged is that the 
catch-up reproduction processing is executed as to the 
particular program. Accordingly, if the catch-up repro- 
duction processing is executed as to all programs, the 
judgment processing of STEP 5604 may be omitted. 
[0082] On the other hand, at STEP 502, if the program 
switching monitoring unit 301 judges that the value of 
the original program memory unit is not null, it also judg- 
es whether the value is the program number of the par- 
ticular program (STEP 506). If the program switching 
monitoring unit 301 judges that the value is the particular 
program number, it controls the selector 302 to send out 
a video/audio stream which is sent out from the demul- 
tiplexer 203, directly to the storing device 208. Thus, the 
selector 302 is made to select a terminal c set forth in 
Fig. 3. On the other hand, at STEP 506, if the program 
switching monitoring unit 301 judges that the value is 
the particular program number, it controls to execute a 
storage/catch-up reproduction processing. That is, the 
program switching monitoring unit 301 controls the se- 



lector 302 to send out a video/audio stream which is di- 
rectly sent out from the demultiplexer 203 to the VA data 
decoder 207a. Thus, the selector 302 is made to select 
a terminal b set forth in Fig. 3. At the same time, the 
5 program switching monitoring unit controls to send out 
a video/audio stream which was read out from the stor- 
ing device 208, instead of the video/audio stream which 
is sent out from the demultiplexer 203, to the VA data 
decoder 207a (STEP 508). 
w [0083] Fig. 6 is a flow chart for explaining the detail of 
the processing at STEP 508. As shown in this figure, the 
real time reproduction position obtaining unit 304 ob- 
tains a position to be reproduced in the real time axis by 
TOT (Time Offset Table) or TDT (Time Data Table) from 
15 psi of TS which passes through itself and informs the 
time difference detecting unit 306 of it. On the other 
hand, the read-out reproduction position obtaining unit 
305 obtains a reproduction position in the reproduction 
time axis of a video/audio stream which passes through 
itself and informs the time difference detecting unit 306 
of it (STEP 601). 

[0084] A calculation of a time signal which is used in 
the time difference detecting unit 306 differs depending 
upon a storing data format. In the time of storing, PSI is 
separated from TS and only MPEG-TS is stored, it is 
preferable to use PCR as a substitute for TOS. Further, 
in case of a format for storing MPEG-PS, it is preferable 
to use ESCR as a time signal. Further, it is possible to 
use a time signal based upon NPT (Normal Play Time). 
[0085] The time difference detecting unit 306 com- 
pares the informed reproduction position in the real time 
axis to the reproduction position in the reproduction time 
axis and judges whether there is a time difference be- 
tween them (STEP 602). If the time difference detecting 
unit 306 judges that there was the time difference, it 
gives an instruction for executing the catch-up reproduc- 
tion processing to the catch-up reproduction processing 
unit 307, accordingly, the catch-up reproduction 
processing unit 307 executes (STEP 603). The catch- 
up reproduction processing unit, as long as it is judged 
that there is a time difference, executes the catch-up re- 
production. On the other hand, a time difference sensing 
unit 703, in case that it is judged that the time difference 
is solved, that is, the reproduction position in the repro- 
duction time axis caught up with the reproduction posi- 
tion in the real time axis by the catch-up reproduction 
processing, controls the selector 302 to send out a vid- 
eo/audio stream to the VA data decoder 207a (STEP 
604). Then, the program switching monitoring unit 301 
resets contents of the original program memory 303 
(STEP 605). In addition, in case that while the stream 
control unit206 is executing the storage/catch-up repro- 
duction processing, furthermore, that a program is 
switched is detected, for example, by an interruption 
processing, the stream control unit 206 stops storage/ 
reproduction processing. 

[0086] Then, an operation example of the receiving 
terminal device which is configured as described above 
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will be explained with reference to Fig. 7 and Fig. 8. Fig. 
7 is a view for explaining a condition of program switch- 
ing when a program is viewed as a stream. In this figure, 
contents to be viewed by viewers are shown by a solid 
line arrow and contents to be viewed during the catch- 
up reproduction processing are shown by a wavy line 
arrow. Fig. 8 is a view showing transition of a screen 
which is displayed on a display. 
[0087] Firstly, it is assumed that a viewer selects a real 
time program A of a broadcasting station which is con- 
ducting program providing services to which the stor- 
age/catch-up reproduction is possible. (A screen at this 
time is shown in Fig. 8A(a). Hereinafter, same as 
above). It is assumed that a data broadcasting program 
B is one which realizes an interlock type data broadcast- 
ing service of the program A and transferred in advance 
by the Carousel system and stored in the storing device 
208. 

[0088] Now, it is assumed that while the main program 
content is being viewed, program switching E1 occurred 
and a screen was switched to the data broadcasting pro- 
gram B (the same figure (b)). The program switching E1 
occurs by, for example, fire by an event message which 
is transferred by a data broadcasting channel and a pro- 
gram switching operation by a viewer. The receiving ter- 
minal device starts a video recording of the program A 
which was switched, that is, storage of its video/audio 
stream. Assuming that after lapse of predetermined 
time, program switching E2 occurred and the program 
A was selected again, the receiving terminal device 
starts reproduction of the program A which was stored. 
In this time, the receiving terminal device, in order to 
solve the time difference which was occurred by the in- 
terruption, executes the catch-up reproduction process- 
ing concerning CM1 and CM2 (the same figure (c)). The 
receiving terminal device displays the time difference on 
the screen, while the catch-up reproduction processing 
is executed, in order to present viewers how much time 
difference is occurring. The catch-up reproduction 
processing is executed by for example, thin-out CM1 
and CM2 themselves. Then, the receiving terminal de- 
vice judges that even when CM1 and CM2 are thinned 
out, the time difference still can not be solved and exe- 
cutes out the catch-up reproduction processing con- 
cerning a main content of the program A following CM2 
(the same figure (d) ) . In this case, since it is not pref- 
erable to thin out itself in the same way as CM1 and 
CM2, the receiving terminal device executes fast play 
reproduction with the extent that a viewer can recognize 
the contents. The extent (speed) of the fast play repro- 
duction may be changed for example, by remote control 
of a viewer. The receiving terminal device, in case that 
it is judged that the time difference was solved (caught 
up) by the catch-up reproduction processing, returns the 
main content of the program A to normal reproduction 
(the same figure (e)). 

[0089] For example, it is assumed that time when the 
data broadcasting program B is presented for transition 



is 1 minute and 15 seconds. Further, it is assumed that 
CM1 and CM2 are of 30 seconds, respectively. In this 
case, the receiving terminal device thins out CM1 and 
CM2 so that the time difference of remaining 1 5 seconds 
5 is made to occur. Assuming that, as the catch-up repro- 
duction processing to the main content, the 10 second 
reproduction in the real time axis is executed for 8 sec- 
onds in the reproduction time axis (20% fast play repro- 
duction), the catch-up is accomplished for 75 seconds. 
10 [0090] In the operation example, it was explained that 
commercial programs CM1 and CM2 were thinned out 
and the catch-up reproduction processing is applied to 
a portion of the time difference which could not be ap- 
propriated by the fast play reproduction to the main con- 
15 tent but the fast play reproduction may be applied to 
CM1 and CM2 and only extent of start several seconds 
may be reproduced, respectively. Furthermore, the fast 
play reproduction may not be applied to the main con- 
tent but the catch-up reproduction processing may be 
applied only to the commercial programs. That is, to the 
main content, the catch-up reproduction (time difference 
reproduction) by normal reproduction may be applied 
and every time that a commercial program is broadcast, 
the catch-up reproduction processing may be applied to 
the commercial program until the time difference is 
solved. 

[0091] As described above, according to the embod- 
iment, when a particular program which is being viewed 
by a viewer is switched to another program, storage of 
the stream corresponding to the program is started and 
when the switched program is switched to the former 
program, the stored stream is reproduced and contents 
of the program A which was interrupted can be viewed 
without fail. 

[0092] Further, according to the embodiment, since 
the catch-up reproduction processing is executed on the 
occasion of reproducing the stream which was stored, 
the time difference in the real time axis can be solved. 
[0093] Accordingly, a viewer can view a real time pro- 
gram, without recognizing a video recording reserva- 
tion, with enjoying a program switching. 

[Second Embodiment] 

[0094] This embodiment relates to a modified exam- 
ple of the above-described storage/catch-up reproduc- 
tion processing, and more specifically, is characterized 
in that a stream (packet) which is judged to be unnec- 
essary because of the catch-up reproduction process- 
ing is not revoked but stored again in the storing device 
208 so that a viewer can view it by reading out and re- 
producing it at arbitrary timing. 
[0095] Fig. 9 is a block diagram showing a structure 
of the stream controlling unit 206 according to the em- 
bodiment. In this figure, the same symbols are applied 
to the same element for realizing functions as in the 
above embodiment. 

[0096] As shown in the figure, the stream controlling 
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unit 206 further comprises a read-out controlling unit 
901 and a selector 902, and the catch-up reproduction 
processing unit 307' is configured so that it can send out 
again a video/audio stream which became an object for 
the catch-up reproduction processing to the storing de- 5 
vice 208. In addition, it is preferable that the catch-up 
reproduction processing unit 307', on the occasion of 
storing again in the storing device 208, stores with re- 
spect to each category which the program contents be- 
long to. Such category can be specified with reference to 
to LIT. Further, filtering information in accordance with 
preference of a viewer may be set in advance and pro- 
gram contents which are judged to be necessary may 
be stored in the storing device 208 in accordance with 
this filtering information. The filtering information is in- 15 
putted through for example, the input unit 201 and 
stored in a predetermined memory area. 
[0097] The read-out control part 901 , when it receives 
an instruction for video recording and reproducing 
through the input unit 204 from a viewer, controls the 20 
selector 902 to directly send out a video/audio steam 
which is read out from the storing device 208 to the VA 
data decoder 207a. At this time, it is preferable that the 
receiving terminal device stops receiving by the tuner 
201. Thus, the receiving terminal device, in the same 25 
way as normal video recording and reproducing, can 
read out the stream which was stored again in the stor- 
ing device 208 and can reproduce it. Accordingly, a 
viewer, for example in case of convenient time for him/ 
her, can view only commercial programs. 30 

[Third Embodiment] 

[0098] This embodiment relates to a modified exam- 
ple of the above-described storage/catch-up reproduc- 35 
tion processing, and more specifically, is characterized 
in that in a particular program, when a certain program 
content is broadcast for the second time or more, thin- 
out is executed to the second or subsequent broadcast 
program content in the receiving terminal device. In oth- *o 
er words, in case that a portion which is stored contains 
the same program content, reproduction is executed by 
avoiding overlap of the content. 
[0099] Fig. 1 0 is a block diagram showing a structure 
of the stream controlling unit 206 according to the em- 45 
bodiment. In this figure, the same symbols are applied 
to the same means for realizing functions as in the 
above embodiment. 

[0100] As shown in the figure, the stream controlling 
unit 206 further comprises a reproduction program con- so 
tent memory 1001. The reproduction program content 
memory 1001 stores information for identifying a pro- 
gram content corresponding to a stream which is an ob- 
ject for the catch-up reproduction processing and which 
has been already sent out to the VA data decoder 207a. 55 
A catch-up reproduction processing unit 1002 refers to 
the reproduction program content memory 1001 and 
judges as the same program content as the program 



content which had been already reproduced, executes 
the catch-up reproduction processing concerning a vid- 
eo/audio stream which is being read out. 
[0101] Fig. 11 is a flow chart for explaining an opera- 
tion of the catch-up reproduction processing unit 1002 
according to the embodiment. As shown in this figure, 
the catch-up reproduction processing unit 1 002, as to a 
video/audio stream which is being read out from the 
storing device 208, judges whether it is a program to be 
catch-up reproduced or not (STEP 1101). In this embod- 
iment, it is assumed that a particular program content 
such as a commercial program and so on is an object 
for the catch-up reproduction. If the catch-up reproduc- 
tion processing unit 1102 judges that the program con- 
tent is the object for catch-up reproduction, judges 
whether it is no yet reproduced even once, that is, it is 
an initial reproduction or not (STEP1102).The catch-up 
reproduction processing unit 1002 judges as the initial 
reproduction, writes identification information of its pro- 
gram content in the reproduction program content mem- 
ory 1 001 (STEP 1 1 03), and directly sends out the video/ 
audio stream to the VA data decoder 207a (STEP 1 1 04). 
On the other hand, when the catch-up reproduction 
processing unit 1002 judges that the program content 
was already reproduced, that is, the program is repro- 
duced for the second time or more, because the identi- 
fication information is recorded in the reproduction pro- 
gram content memory 1 001 , the catch-up reproduction 
processing unit 1002 executes the catch-up reproduc- 
tion processing on a video/audio stream that is read out 
(STEP1105). In case that a program content is not the 
object for catch-up reproduction, for example is content 
of the main program, the catch-up reproduction 
processing unit 1001 sends out the video/audio stream 
to the VA data decoder 207a. 
[0102] Next, an operation example of the receiving 
terminal device according to the embodiment will be ex- 
plained by use of Fig. 1 2. Fig. 1 2 is a view for explaining 
a condition of program switching when a program is 
viewed as a stream. In this figure, a content which is 
viewed by a viewer is shown by a solid line arrow and a 
content which the catch-up reproduction processing is 
applied to is shown by a wavy line arrow. 
[0103] It is assumed that a viewer selects a real time 
program A of a broadcasting station which is conducting 
a program providing service which the storage/catch-up 
reproduction can be applied to. Further, it is assumed 
that a program B is also a real time program. 
[0104] Now, it is assumed that while, the commercial 
program CM1 is being viewed following the main pro- 
gram content, the program switching E1 occurs and it is 
switched to the program B. The program switching E1 
occurs by, for example, a fire triggered by an event mes- 
sage which is transferred by a data broadcasting chan- 
nel, or a program switching operation conducted by a 
viewer. The receiving terminal device starts a video re- 
cording of the program A which was switched, that is, 
storage of its video/audio stream. Assuming that after 
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lapse of predetermined time, program switching E2 oc- 
curred and the program A was selected again, the re- 
ceiving terminal device starts reproduction of the pro- 
gram A which was being stored. The receiving terminal 
device, since CM2 has not yet been viewed, reproduces 
it as-is, and thereafter reproduces the main content and 
CM3. When the receiving terminal device comes to the 
point where the CM1 is just being reproduced, it thins 
out (skips) the CM1 because it judges that the CM1 was 
already viewed. Thus, it is possible to solve the time dif- 
ference by a time portion which is required for reproduc- 
tion of CM1 . The receiving terminal device, after it skips 
CM1 , reproduces the main program content. In the em- 
bodiment, even by the thin-out of CM1, the time differ- 
ence which is generated by the interruption can not be 
solved. Then, it is preferable that the receiving terminal 
device, in case that CM2 and CM3 are broadcast again, 
solves the time difference by thinning out them. Further, 
a commercial program may be reproduced in fast play. 
[0105] As described above, according to the present 
embodiment, when a certain program content is broad- 
casted for the second time or more in a particular pro- 
gram, the reproduction of that program content which is 
broadcasted for the second time or more will be omitted. 
Accordingly, the viewer will be forced to view the same 
commercial program, etc., repeatedly. 

[Fourth Embodiment] 

[01 06] This embodiment relates to a modified exam- 
ple of the above-described storage/catch -up reproduc- 
tion processing. More particularly, this embodiment is 
characterized in that, a main program content in a par- 
ticular program is displayed on a first window (screen) 
and another program content as an object for catch-up 
reproduction is displayed on a second window. 
[0107] For example, as shown in Fig. 13(a), the re- 
ceiving terminal device divides a screen area which is 
displayed on a display 211a into a screen W1 and a 
screen W2, and reproduces a program content on re- 
spective screens. Such a viewing mode is called as a 
multiscreen. In this figure (a), a program content of the 
main program is reproduced on the screen W1 and a 
commercial program in the stored program contents is 
reproduced on the screen W2. Further, as shown in this 
figure (a), the receiving terminal device may divide the 
screen are into 3 areas. In this case, it is preferable that 
an aspect ratio of the screen is converted from 16:9 to 
4:3. 

[0108] Fig. 14 is a block diagram showing a structure 
of the stream controlling unit 206 according to the em- 
bodiment. In addition, in this figure, the same symbols 
are applied to the same means for realizing functions as 
in the above embodiment. 

[0109] As shown in the figure, the stream controlling 
unit 206 sends out a video/audio stream corresponding 
to a main program content to be displayed on the first 
screen and a video/audio stream corresponding to an- 



other program content to be displayed on the second 
screen to the VA data decoder 207a, respectively. That 
is, as to a main program content, the stream controlling 
unit 206 sends out a video/audio stream which was read 

5 out from the storing device 208 to the VA data decoder 
207a in the same manner as in the above embodiment 
until the time difference is solved, and at the time point 
that the time difference is solved, sends out a video/au- 
dio stream which is sent out from the demultiplexer 203 

10 to the VA data decoder 207a. On the other hand, as to 
a content of another program such as a commercial pro- 
gram, etc., other than a main program, the stream con- 
trolling unit 206 sends out the video/audio stream which 
was read out from the storing device 208 separately to 

15 the VA data decoder 207a. More specifically, a catch-up 
reproduction processing unit 1401 judges that a video/ 
audio stream which was read out from the storing device 
208 corresponds to a main program or not, and when it 
is judged as the video/audio stream corresponding to a 

20 program other than a main program, for example, a 
commercial program, sends out to the VA data decoder 
207a separately from the video/audio stream corre- 
sponding to the main program content. Thus, since the 
stream control part 206 is possible to send out a more 

25 video/audio stream from the storing device 208, the time 
difference can be solved. 

[Fifth Embodiment] 

30 [0110] This embodiment relates to a modified exam- 
ple of the above-described storage/catch-up reproduc- 
tion processing. More specifically, the embodiment is 
characterized in that at a time point that a program is 
switched, position information (reproduction start posi- 
35 tion information) by which reproduction of a stream is to 
be resumed is set and stored in advance, and when it 
is returned to the program, reproduction is resumed 
from the stream specified by the reproduction start po- 
sition information. 
40 [0111] Fig. 15 is a block diagram showing a structure 
of the stream control unit 206 according to the embodi- 
ment. In this figure, the same symbols are applied to the 
same means for realizing functions as in the above em- 
bodiment. 

45 [01 1 2] In the embodiment, in order to simplify an ex- 
planation, it is assumed that a video/audio stream is all 
sent out through the storing device 208 to the VA data 
decoder 207a. That is, in the initial state, a video/audio 
stream is stored through the terminal c of the selector 
50 302 in the storing device 208 and read out from the stor- 
ing device 208 and sent out through the selector e to the 
VA decoder 207a. 

[0113] When the program switching monitoring unit 
301 judges that the initial state is transferred to the first 
55 state, the program switching monitoring unit 301 stores 
program number of the program A and reproduction 
start position information into the original program mem- 
ory 303. In the embodiment, it is assumed that an event 
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ID of LIT which is defined as PSI/SI is used as the re- 
production start position information. The event ID is 
sent outfrom eitherthe output controlling unit 21 0 based 
upon fire of an event message or sent out from the de- 
multiplexer 203. When the program switching monitor- s 
ing unit 301 judges that the first state is transferred to 
the second state, it controls the selector 302 to select 
the terminal b and controls a selector 1501 to select a 
terminal d, respectively, and informs a read-out search- 
ing unit 1501 of the event ID which was stored in the 10 
original program memory 303. 
[0114] The read-out searching unit 1502 reads out a 
video/audio stream (packet) after the event ID from the 
storing device 208 in accordance with the event ID 
which was informed, and sends out it to the read-out '5 
reproduction position obtaining unit 305. Thus, it be- 
comes possible to read out from the video/audio stream 
at the position which was designated by the event ID 
and to reproduce it. 

[0115] Thecatch-up reproduction processing unit 307 20 
executes catch-up reproduction processing concerning 
the video/audio stream which was read out, in the same 
way as in the above embodiment. Then, the time differ- 
ence detecting unit 306, at the time point that the time 
difference was solved, controls the selector 302 and the 25 
selector 1601. 

[0116] In addition, an event relation table ERT (Event 
Relation Table) etc. may be used as the reproduction 
start position information. Further, sequence number of 
a packet may be used as the reproduction start position so 
information. Further, time management information at 
the time point of program switching maybe obtained and 
reproduction of a stream may be executed from the po- 
sition which is shown by the time management informa- 
tion. 35 
[01 17] Furthermore, the reproduction start position in- 
formation is not limited to the time point that a program 
was switched. For example, it may be a head position 
of a certain program content (for example, commercial 
program) which is included in a main program. 40 
[0118] Next, an operation example of the receiving 
terminal device configured as described above accord- 
ing to the embodiment will be explained by use of Fig. 
16. Fig. 16 is a view for explaining a condition of program 
switching when a program is viewed as a stream. 45 
[01 1 9] Firstly, it is assumed that a viewer is selecting 
a real time program A of a broadcasting station which is 
conducting a program providing service in which stor- 
age/catch-up reproduction is possible. In addition, it is 
assumed that a data broadcasting program B is one so 
which realizes a interlock type data broadcasting serv- 
ice of the program A and transferred in advance by the 
Carousel system and stored in the storing device 208. 
[0120] Now, it is assumed that while a main program 
content is being viewed, program switching E1 occurs 55 
and a screen is switched to the data broadcasting pro- 
gram B (FIG. 16B). The program switching E1 occurs 
by fire of an event message which is transferred in a 



data broadcasting channel or a program switching op- 
eration by a viewer. After lapse of a predetermined time, 
if the program switching E2 occurred or the program A 
was selected again, the receiving terminal device starts 
reproduction of the program A which is stored in accord- 
ance with the reproduction start position information. It 
is assumed that, the reproduction start position informa- 
tion M2 is designated in advance by an instruction from 
the broadcasting station. At this time, the receiving ter- 
minal device executes the catch-up reproduction 
processing concerning CM2, in order to solve the time 
difference which was generated by the interruption. The 
receiving terminal device, while the catch-up reproduc- 
tion processing is being executed, displays the time dif- 
ference information on the screen, in order to present to 
the viewer how much time difference has occurred. The 
catch-up reproduction processing is executed, for ex- 
ample, on CM2 and the main program content. When 
the receiving terminal device judges that the time differ- 
ence was resolved (caught up) by the catch-up repro- 
duction processing, it reproduces the main program 
content of the program A underthe normal reproduction. 
[0121] In addition, in the embodiment, a position 
which is shown by the reproduction start position infor- 
mation is M2 but M1 before the program switching time 
point may be designated. In this case, CM1 is made to 
be reproduced from the beginning. This means that a 
viewer can view the whole CM1 . 

[Other Embodiment] 

[0122] The above-described respective embodi- 
ments are examples for explaining the invention, and it 
does not mean that the invention is limited only to these 
embodiments. 

[0123] The invention can be practiced in various em- 
bodiments as exemplified below as long as not deviated 
from the spirit of the invention. 

(1) In the above-described embodiments, as caus- 
es of program switching, it is explained about one 
by a viewer himself/herself and/or one by an event 
message, but it may be one by effect of reservation 
using EPG and one by a link descriptor and a hyper 
link descriptor. Further, switching may be executed 
by executing launchDocument() and an applet of 
scripts in contents data. These program switching 
is detected by a script which parses an instruction 
from the input unit or detected by a script for chang- 
ing a service ID and/or PID. 

(2) The receiving terminal device may be configured 
to store as an entire transport stream during the 
stream is stored. Further, as to a stream to which a 
scramble is applied to, it is preferable to store it after 
it was descrambled. 

(3) The receiving terminal device may be configured 
to be able to select whether contents such as a flash 
report according absolute time is displayed or not, 
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on the occasion of the catch-up reproduction 
processing, by an instruction of a viewer. 

(4) In the above-described embodiments, it is ex- 
plained about the switching between a real time 
type program and a data broadcasting program but s 
it is not stick upon this combination in particular. An 
embodiment for switching between real time type 
programs, between data broadcasting programs is 
included in the aspect of the invention. Further, the 
first stream may be a real time type program and 
the second stream may be a program which was 
video recorded. 

In case of the switching between the above-de- 
scribed real time programs, a plurality of receiving 
functions are required. On the other hand, in case 
of the switching between data broadcasting pro- 
grams, a function to store multiple Carousel data in 
plurality is required. These required functions can 
be implemented in response to the above switching 
need. Forexample, if there is a request for switching 
real time programs A, B, and C, one by one, at least 
three receiving functions are required. On the other 
hand, if an operation is set to execute transition only 
in the same TS at a broadcasting station side, only 
one receiving function may be required. Further, if 
the program after the switch must be returned to the 
original program by all means even in case of 
switching by an operation of the viewer, only two 
receiving functions may be required. 

(5) In the above-described embodiments, it is as- 30 
sumed that, when a viewer switched the program A 

to the program B, video recording is executed to the 
program A uniformly but, the video recording may 
be executed under selection of the viewer. That is, 
the viewer operates a first operation to switch from 35 
the program A to the program B, video recording to 
the program A maybe executed, and when the view- 
er operates a second operation, video recording to 
the program A may not be executed. The first and 
second operations are judged by, for example, sort 40 
of a button which is pressed down. 

(6) When the program selection is not returned back 
to the program A for a predetermined time after the 
selection is switched from the program A to the pro- 
gram B and the receiving terminal device has start- 45 
ed the video recording of the program A, the receiv- 
ing terminal device may stop the video recording of 
the program A. 

(7) The video recording that starts when the pro- 
gram selection is switched from the program A to so 
the program B may be designed such that it records 
only the program A. That is, the receiving terminal 
device stops the video recording at the time when 

the program A ends; it does not record another pro- 
gram which is broadcasted following the program 55 
A. Whether broadcasting of a program ends or not 
is judged with reference to EIT (Event Information 
Table). 



[0124] As stated above, according to the invention, it 
becomes possible to provide a broadcasting service 
which does not cause a viewing failure of a program con- 
tent, even in case that a switching of program selection 
is executed. 



Claims 



3. The receiving terminal device according to Claim 2, 
characterized in that said receiving terminal de- 
vice stops the video recording of the first program 
when the position of reproducing the first program 
catches up with the position in which the first pro- 
gram is being broadcasted by the broadcasting sta- 
tion. 

4. The receiving terminal device according to Claim 2 
or Claim 3, characterized in that said receiving ter- 
minal device thins out a commercial program in the 
first program. 

5. The receiving terminal device according to Claim 2 
or Claim 3, characterized in that, when two or 
more commercial programs of a predetermined 
content are included in the recorded first program, 
said receiving terminal device provides the com- 
mercial program once. 

6. A receiving terminal device which selects a partic- 
ular stream from a received steam and reproduces 
the selected stream as video/audio information, 
characterized in that, when a second stream is se- 
lected while a first stream is being filtered through, 
said receiving terminal device stores a portion of the 
first stream following the time point at which the sec- 
ond stream is selected, and reproduces the second 
stream. 

7. The receiving terminal device according to Claim 6, 



*o 1. A receiving terminal device characterized in that, 
when a first program which is being provided is 
switched to a second program, said receiving ter- 
minal device provides the second program and 
starts a video recording of the first program, and, 
*5 when the switched second program is switched 
back to the first program, provides the recorded first 
program. 

2. The receiving terminal device according to Claim 1 , 
20 characterized In that, when the first program is se- 
lected again, said receiving terminal device exe- 
cutes a catch-up reproduction of the first program 
in the manner such that the position of reproducing 
the first program catches up with the position in 
25 which the first program is being broadcasted by a 
broadcasting station. 
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characterized in that, when the first stream is se- 
lected again after the second stream is selected, 
said receiving terminal device reproduces the 
stored stream. 

5 

8. The receiving terminal device according to Claim 7, 
characterized in that said receiving terminal de- 
vice obtains a reproduction start position informa- 
tion from the received stream and reproduces the 
stored stream from the reproduction start position. io 

9. The receiving terminal device according to Claim 7, 
characterized in that said receiving terminal de- 
vice obtains time management information in the 
selected time point and reproduces the stored 15 
stream from the position specified in the obtained 
time management information. 

10. The receiving terminal device according to Claims 

7 to 9, characterized In that, when the first stream 20 
is selected again, said receiving terminal device re- 
produces the stored stream in the manner that a re- 
production position of the stored stream in a repro- 
duction time axis catches up with a reproduction po- 
sition of the first stream in a real time axis. 25 



12, 

characterized in that, when the stored stream in- 
cludes n partial streams corresponding to a prede- 
termined program content, said receiving terminal 
device thins out n-1 partial streams. 

16. The receiving terminal device according to Claim 
13, 

characterized In that said receiving terminal de- 
vice stores the thinned out partial streams. 

17. The receiving terminal device according to Claim 
10, 

characterized in that said receiving terminal de- 
vice stores the first stream until the reproduction po- 
sition of the stored stream in the reproduction time 
axis catches up with the reproduction position of the 
first stream in the real time axis. 

18. The receiving terminal device according to Claim 6, 
characterized in that, when the first stream is se- 
lected again after said second stream is selected, 
said receiving terminal device reproduces the se- 
lected first stream on a first window and reproduces 
the stored stream on a second window. 



10 



11. The receiving terminal device according to Claim 
10, 

characterized in that said receiving terminal de- 
vice displays on a screen a time difference informa- 30 
tion between the reproduction position of the stored 
stream in the reproduction time axis and the repro- 
duction position of the first stream in the real time 
axis. 

35 

12. The receiving terminal device according to Claim 
10, 

characterized in that said receiving terminal de- 
vice reproduces the stored stream while a thin-out 
processing to the stored stream. 40 

13. The receiving terminal device according to Claim 
12, 

characterized in that said receiving terminal de- 
vice executes a thin-out processing on a partial 45 
stream corresponding to a predetermined program 
content in the stored stream. 

14. The receiving terminal device according to Claim 

12, ^50 
characterized in that said receiving terminal de- 
vice executes a thin-out processing of a partial 
stream corresponding to a program content other 
than a program content in the stored stream which 
is selected in accordance with user information in- 55 
putted in advance. 

15. The receiving terminal device according to Claim 



19. The receiving terminal device according to Claim 
18, 

characterized in that said receiving terminal de- 
vice reproduces the partial stream corresponding to 
the first program content in the stored stream on the 
second window. 

20. The receiving terminal device according to Claim 
10, 

characterized in that said receiving terminal de- 
vice executes fast play reproduction of the partial 
stream corresponding to the second program con- 
tent in the stored stream on the first window. 

21 . The receiving terminal device according to Claim 6, 
characterized in that said receiving terminal de- 
vice selects the second stream based upon pro- 
gram selection information in the received stream. 

22. The receiving terminal device according to Claim 6, 
characterized in that said receiving terminal de- 
vice selects the second stream based upon pro- 
gram selection information supplied from a viewer. 

23. The receiving terminal device according to Claim 6, 
characterized in that said receiving terminal de- 
vice selects the second stream based upon sched- 
ule information set based upon program guide in- 
formation. 

24. A receiving terminal device comprising: 



14 



27 



EP1 255 404 A1 



28 



a stream receiver for receiving a transferred 
stream, 

a storing unit for storing the received stream, 
an accepting device for receiving program se- 
lection information for selecting a program to 
be provided to a viewer, 
a selector for selecting a particular stream in 
the received stream based upon the program 
selection information, 

an output unit for outputting video/audio infor- 
mation based upon a stream to be transmitted, 
and 

when the program selection information for se- 
lecting a second stream is received while a first 
stream is being filtered through, a controller for 
outputting a stream in the first stream following 
the time point at which the second stream is se- 
lected to the storing unit. 

25. The receiving terminal device according to Claim 
24, 

wherein the controller, when the program selection 
information for selecting the first stream is received 
after the second stream is selected, reads out a 
stream stored in the storing unit and transmits the 
read out stream to the output unit. 

26. The receiving terminal device according to Claim 
25, 

wherein the controller includes a reproduction start 
position management unit for managing a repro- 
duction start position of the stream transmitted to 
the output unit. 

27. The receiving terminal device according to Claim 
26, 

wherein the reproduction start position manage- 
ment unit outputs the stream to the output unit from 
a position specified in the reproduction start position 
information allocated to the stream to be outputted 
to the storing device. 

28. The receiving terminal device according to Claim 
26, 

wherein the reproduction start position manage- 
ment unit includes a switching time management 
unit for obtaining time management information at 
the time of receiving program switching information 
by the accepting unit, and wherein the reproduction 
start position management unit outputs the stream 
to the output unit from a position specified in the ob- 
tained time management information. 

29. The receiving terminal device according to Claim 
25, 

wherein the reproduction start position manage- 
ment unit reproduces a stream outputted from the 
storing unit to the output unit in the manner that the 



reproduction position of the stream outputted from 
the storing unit to the output unit catches up with 
the reproduction position of the first stream to be 
outputted from the receiver to the output unit. 

5 

30. The receiving terminal device according to Claim 
25, 

wherein the controller outputs the stored stream to 
the storing device until the reproduction position of 
the stream which is outputted from the storing unit 
to the output unit catches up with the reproduction 
position of the first stream to be outputted from the 
receiver to the output unit. 

*s 31. The receiving terminal device according to Claim 
25, 

wherein the controller, when the first stream is se- 
lected again after the second stream is selected, 
transmits the first stream which is transmitted from 
the receiver and the stream which is read out from 
the storing unit to the output unit, respectively. 

32. The receiving terminal device according to Claim 
24, 

wherein the accepting unit detects program switch- 
ing information included in the received stream. 

33. A program providing method comprising steps of: 

switching a first program to a second program 
while the first program is being provided, 
video recording of the first program while the 
switched second program is being provided, 
switching back to the first program while the 
switched second program is being provided, 
and 

providing the first program which is being re- 
corded. 

34. The program providing method according to Claim 
33, 

wherein when the first program is switched back, 
the recorded first program is executed in the man- 
ner such that the a position of recording and repro- 
ducing thefirst program catches up with the position 
in which the first program is being broadcasted by 
a broadcasting station. 

35. The program providing method according to Claim 
34, 

wherein the first program is recorded until the posi- 
tion of recording and reproducing the first program 
catches up with the position in which the first pro- 
gram is being broadcasted by the broadcasting sta- 
tion by the catch-up reproduction. 

36. A control method of a receiving terminal device in 
which a particular stream in a received stream is 



20 



35 



40 



45 



50 



15 



29 



EP 1 255 404 A1 



selected, and the selected stream is reproduced as 
video/audio information, 

wherein when a second program is selected while 
a first stream is being filtered through, a stream in 
the first stream following a time point at which the s 
second stream is selected is stored and the select- 
ed second stream is reproduced. 

37. The control method of the receiving terminal device 
according to Claim 36, wherein the stored stream 10 
is reproduced when the first stream is selected 
again after the second stream is selected. 
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